Real time detection of metric baseline behavior change

ABSTRACT

Example aspects include techniques for real-time detection of metric baseline behavior change. These techniques may include generating a reference distance signature based on historic time series information for a component metric, the historic time series information corresponding to a first period of time, generating a sample distance signature based on sample time series information for the component metric, the sample time series information corresponding to a second period of time, and comparing the reference distance signature to the sample distance signature to determine a signature difference. In addition, the techniques may include determining that the second period of time is a baseline change candidate based on the signature difference being greater than a distance threshold, and presenting, based at least in part on the signature difference, an alert notification identifying the second period of time as the baseline change candidate.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of U.S. patentapplication Ser. No. 17/390,389 filed Jul. 30, 2021, entitled “Real TimeDetection of Metric Baseline Behavior Change,” which is incorporatedherein by reference in its entirety.

BACKGROUND

A cloud computing environment may provide one or more services tocustomers over a network. Further, the cloud computing environment mayemploy a monitoring service for detecting issues arising within thecloud computing environment. In some instances, anomaly detection may beused to determine when patterns in data do not match an expectedpattern. For example, a monitoring service may perform anomaly detectionover cloud computing performance data associated with the one or moreservices to detect a service issue. However, in some contexts, anundetected service issue may correspond to a baseline change that doesnot include behavior that would qualify as anomalous, e.g., theprobabilistic change in the distribution of a time series may rise orfall without spiking above or below the thresholds used to detectanomalous activity. Detecting baseline changes requires computationinvolving large amounts of historic data. As a result, baseline changedetection is typically performed offline due to high computational costsat scale and/or the inability of traditional approaches to detectbaseline changes with a cadence sufficient for real-time monitoringsolutions.

SUMMARY

The following presents a simplified summary of one or moreimplementations of the present disclosure in order to provide a basicunderstanding of such implementations. This summary is not an extensiveoverview of all contemplated implementations, and is intended to neitheridentify key or critical elements of all implementations nor delineatethe scope of any or all implementations. Its sole purpose is to presentsome concepts of one or more implementations of the present disclosurein a simplified form as a prelude to the more detailed description thatis presented later.

In an aspect, a method may include generating a reference distancesignature based on historic time series information for a componentmetric, the historic time series information corresponding to a firstperiod of time, generating a sample distance signature based on sampletime series information for the component metric, the sample time seriesinformation corresponding to a second period of time, and comparing thereference distance signature to the sample distance signature todetermine a signature difference. The method may further includedetermining that the second period of time is a baseline changecandidate based on the signature difference being greater than adistance threshold, and presenting, based at least in part on thesignature difference, an alert notification identifying the secondperiod of time as the baseline change candidate.

In another aspect, a device may include a memory storing instructions,and at least one processor coupled to the memory and to execute theinstructions to: generate a reference distance signature based onhistoric time series information for a component metric, the historictime series information corresponding to a first period of time,generate a sample distance signature based on sample time seriesinformation for the component metric, the sample time series informationcorresponding to a second period of time, compare the reference distancesignature to the sample distance signature to determine a signaturedifference, determine that the second period of time is a baselinechange candidate based on the signature difference being greater than adistance threshold, and present, based at least in part on the signaturedifference, an alert notification identifying the second period of timeas the baseline change candidate.

In another aspect, an example computer-readable medium storinginstructions for performing the methods described herein and an exampleapparatus including means of performing operations of the methodsdescribed herein are also disclosed.

Additional advantages and novel features relating to implementations ofthe present disclosure will be set forth in part in the description thatfollows, and in part will become more apparent to those skilled in theart upon examination of the following or upon learning by practicethereof.

BRIEF DESCRIPTION OF THE DRAWINGS

The Detailed Description is set forth with reference to the accompanyingfigures, in which the left-most digit of a reference number identifiesthe figure in which the reference number first appears. The use of thesame reference numbers in the same or different figures indicatessimilar or identical items or features.

FIG. 1 is a diagram showing an example of a cloud computing system, inaccordance with some aspects of the present disclosure

FIG. 2A illustrates an example graphical representation of a baselinechange, in accordance with some aspects of the present disclosure.

FIG. 2B illustrates an example graphical representation of a referencedistance signature and a sample distance signature, in accordance withsome aspects of the present disclosure.

FIG. 3 is a flow diagram illustrating an example method for evaluatingpeer groups for comparative anomaly, in accordance with some aspects ofthe present disclosure.

FIG. 4 is a block diagram illustrating an example of a hardwareimplementation for a cloud computing device(s), in accordance with someaspects of the present disclosure.

DETAILED DESCRIPTION

The detailed description set forth below in connection with the appendeddrawings is intended as a description of various configurations and isnot intended to represent the only configurations in which the conceptsdescribed herein may be practiced. The detailed description includesspecific details for the purpose of providing a thorough understandingof various concepts. However, it will be apparent to those skilled inthe art that these concepts may be practiced without these specificdetails. In some instances, well-known components are shown in blockdiagram form in order to avoid obscuring such concepts.

This disclosure describes techniques for real-time detection of baselinechange of a cloud component metric. In particular, aspects of thepresent disclosure provide a baseline monitoring module configured toperiodically generate a reference signature representing a historicbaseline for a component metric and a sample signature representingrecently-collected measurements of the component metric, and utilize thedifference between the signatures to detect baseline change for thecomponent metric within a cloud computing environment. As used herein,in some aspects, a component metric may refer to one or moremeasurements associated with activity of a component and/or user of thecloud computing environment. Accordingly, for example, a cloud computingenvironment may employ the baseline monitoring module to performdetection of probabilistic change in the distribution of measurements ofa cloud component metric at the cadence needed for at least near-realtime monitoring in a cloud computing environment, which may be used toimprove system performance and/or reduce time to mitigate serviceincidents.

In accordance with some aspects of the present disclosure, a baselinemonitoring module is configured to accurately and efficiently detectbaseline change based on signature dissimilarity in at least near-realtime, e.g., less than ten minutes. For example, the baseline monitoringmodule may be configured to employ one-class support vector model (SVM)to generate a reference signature representing the baseline of a cloudcomponent metric and periodically generate sample signaturesrepresenting measurements of the cloud component metric over a recentperiod of time. Further, the baseline monitoring module may detect abaseline change when a dissimilarity measurement between the referencesignature and a sample signature is greater than a preconfiguredthreshold. By using signatures generated with minimal computational costto identify baseline changes, the baseline monitoring module provides anefficient monitoring process that can monitor for baseline changes at acadence that can be employed for real-time monitoring within a cloudcomputing environment.

Illustrative Environment

FIG. 1 is a diagram showing an example of a cloud computing system 100,in accordance with some aspects of the present disclosure. Asillustrated in FIG. 1 , the cloud computing system 100 may include acloud computing platform 102, a plurality of client devices 104(1)-(n)associated with a plurality of clients 106(1)-(n), and a plurality oftenant devices 108(1)-(n) associated with a plurality of tenants110(1)-(n). The cloud computing platform 102 may be a multi-tenantenvironment that provides the client devices 104(1)-(n) with access toapplications, services, files, and/or data via one or more network(s)112. In particular, the cloud computing platform 102 may implement amulti-tenant architecture wherein the resources 114(1)-(n) of the cloudcomputing platform 102 are shared among the tenants 110(1)-(n) butindividual data associated with each tenant 110 is logically separated.As described herein, the tenants 110(1)-(n) may be customers of thecloud computing platform 102. Further, the tenants 110(1)-(n) may haverelationships with the plurality of clients 106(1)-(n), and provide oneor more tenant components 116(1)-(n) to the plurality of client devices104(1)-(N) via the cloud computing platform 102.

As an example, the tenant component 116(1) may be a website, and theclient device 104(1) may provide a visitor access to the website.Further, the tenant 110(1) associated with the tenant component 116(1)may employ the cloud computing platform 102 to provide features of thewebsite (i.e., tenant component 116(1)) to the client device 104(1). Forinstance, the tenant component 116(1) may configure the cloud computingplatform 102 to transmit the content of the website to the client device104(1) via the network 112. As another example, the tenant component116(2) may be a database instance and the client device 104(1) mayinclude a tenant application that utilizes the database instance via thenetwork 112.

The network(s) 112 may comprise any one or combination of multipledifferent types of networks, such as cellular networks, wirelessnetworks, local area networks (LANs), wide area networks (WANs),personal area networks (PANs), the Internet, or any other type ofnetwork configured to communicate information between computing devices(e.g., the cloud computing platform 102, the client devices 104(1)-(N),the tenant devices 108(1)-(n)). Some examples of the client devices104(1)-(n) and the tenant devices 108(1)-(n) include computing devices,smartphone devices, Internet of Things (IoT) devices, drones, robots,process automation equipment, sensors, control devices, vehicles,transportation equipment, tactile interaction equipment, virtual andaugmented reality (VR and AR) devices, industrial machines, virtualmachines, etc.

Further, each tenant component 116 may be provided via one or moreservices 118 of the cloud computing platform 102. Some examples of theservices 118(1)-(N) include infrastructure as a service (IaaS), platformas a service (PaaS), software as a service (SaaS), database as a service(DaaS), security as a service (SECaaS, big data as a service (BDaaS), amonitoring as a service (MaaS), logging as a service (LaaS), internet ofthings as a service (IOTaaS), identity as a service (IDaaS), analyticsas a service(AaaS), function as a service (FaaS), and/or coding as aservice (CaaS). Further, the resources 114(1)-(n) may be reserved foruse by the services 118(1)-(n). Some examples of the resources114(1)-(n) include computing units, bandwidth, data storage, applicationgateways, software load balancers, memory, field programmable gatearrays (FPGAs), graphics processing units (GPUs), input-output (I/O)throughput, data/instruction cache, physical machines, virtual machines,clusters of virtual machines, clusters of physical machines, etc.Further, the client devices 104(1)-(n) may transmit service requests120(1)-(n) and receive service responses 122(1)-(n) corresponding to theservice requests 120(1)-(n) in order to access the tenant components116(1)-(n).

As illustrated in FIG. 1 , the cloud computing platform 102 may furtherinclude a baseline monitoring module 124 configured to perform real-timebaseline change detection on component metrics collected within thecloud computing platform 102. Some examples of categories of componentmetrics include request duration metrics, dependency duration metrics,and client performance metrics. Further, the aforementioned categoriesof component metrics may be measured for any activity on the cloudcomputing platform 102.

An example of a request duration metric may be the amount of time thatelapses between the receipt of a service request 120 at the tenantcomponent 116(1) and transmission of the service response 122(1) by thetenant component 116(1). An example of a client performance metric maybe an amount of time that elapses during completion of a process by atenant component 116(1) for a type of a client (e.g., a type of browser,operating system, and/or client application). In addition, as describedin detail herein, a baseline change may not be detected by an anomalydetector tuned to detect spikes (i.e., anomalous values) in a componentmetric as no spike would be observed. In addition, traditional baselinechange detection is performed offline and unable to detect baselinechange in real time at scale.

The cloud computing platform 102 may employ the baseline monitoringmodule 124 to periodically determine reference distance signatures126(1)-(n) for the component metrics in a training phase based on themetrics information 128. Further, in a detection phase, the cloudcomputing platform 102 may employ the baseline monitoring module 124 toperiodically determine sample distance signatures 130(1)-(n) for thecomponent metrics in real-time or near-real time, and compare eachreference distance signature 126 to a corresponding sample distancesignature 130 to determine the occurrence of a baseline change withrespect to the component metric.

Further, as illustrated in FIG. 1 , the baseline monitoring module 124may include at least one of a configuration module 132, a metricsmeasurement module 134, a training module 136, a sampling module 138, adetection module 140, and/or an incident management module 142. In someaspects, the configuration module 132 may configure baseline changedetection at the cloud computing platform 102. For example, theconfiguration module 132 may be configured to set the component metricsthat will be monitored by the baseline monitoring module 124, and one ormore parameters for baseline change detection for each component metricat the cloud computing platform 102. In some aspects, the tenant devices108(1)-(n) associated with the tenants 110(1)-(n) and/or the managementdevices 144(1)-(n) associated with an employee of the cloud computingplatform 102 may provide the one or parameters for performing baselinechange detection. For example, the tenant device 108(1) associated withthe tenant component 116(1) may provide one or more parameters formonitoring one or more component metrics of the tenant component 116(1)for baseline change. As another example, the management device 144 mayprovide one or more parameters for monitoring one or more componentmetrics of the resources 114 for baseline change. Further, theconfiguration module 132 may be configured to set the entities (e.g.,the tenant devices 108, management devices 144) that will be notified ofthe detection of a baseline change.

The metrics measurement module 134 may be configured to determine themetrics information 128 representing measurements of the componentmetrics. In some aspects, the metrics information 128 may includerequest duration measurements, dependency duration measurements, and/orclient performance measurements associated with the cloud components(e.g., the resources 114(1)-(n), the tenant component 116(1)-(n), theservices 118(1)-(n), etc.). Further, in some aspects, the metricsmeasurement module 134 may measure a component metric based at least inpart on one or more parameters provided to the configuration module 132.Additionally, or alternatively, in some aspects, the metrics measurementmodule 134 may receive measurements from cloud components associatedwith the measurement and generate the metrics information 128. Inaddition, the metrics measurement module 134 may provide the metricsinformation 128 to the training module 136, the sampling module 138,and/or the detection module 140.

The training module 136 may be configured to periodically generatereference distance signatures 126(1)-(n) for the component metrics ofthe cloud computing platform 102. In some aspects, the training module136 may employ a machine learning technique (e.g., one-class SVM) togenerate the reference distance signatures 126(1)-(n) using historicdata from the metrics information 128 collected over a preconfiguredperiod of time, e.g., the past seven days, and store the referencedistance signatures 126(1)-(n) for use by the detection module 140.

In some aspects, the training module 136 may perform one or morepre-processing steps on the metrics information 128 prior to generatingthe reference distance signatures 126 for the component metrics. Forexample, the training module 136 may identify a time series for acomponent metric within the metrics information 128, deseasonalize thetime series, and/or remove outliers within the time series that arebelow or above preconfigured thresholds. As used herein in some aspects,deseasonalizing may refer to removing or reducing a seasonal componentof a time series. Further, the training module 136 may scale the timeseries to a preconfigured range (e.g., between 0 and 1), and/or removeone or more anomalies and/or outages via an offline algorithm. Further,the training module 136 may employ a one-class SVM with thepre-processed time series. For example, the training module 136 maydetermine a plurality of support vectors via a one-class SVM based onthe pre-processed time series information, compute a matrix of pairwisedistances between each pair of the support vectors, and determine akernel of the matrix. In some aspects, the training module 136 mayemploy gaussian radial basis function (RBF) as the kernel function usedto generate the kernel. As used herein, in some aspects, the kernel of alinear map, also known as the null space or nullspace, is the linearsubspace of the domain of the map which is mapped to the zero vector.

In some aspects, the training module 136 may be further configured todetermine distance thresholds to be employed by the detection module 140to implement baseline change detection for the component metrics. Forexample, the training module 136 may be configured to partition a timeseries for a component metric of a cloud component into a plurality oftime series windows, and determine a distance score for each of theplurality of time series windows. In some aspects, each distance scorerepresents a distance between an individual time series window and theother plurality of time series windows. Further, the training module 136may determine the distance threshold by generating a vector of thedistance scores and applying a three-sigma calculation to the vector toidentify an abnormal score for the time series as the distancethreshold.

The sampling module 138 may be configured to periodically generatesample distance signatures 130(1)-(n) for the component metrics of thecloud computing platform 102. In some aspects, the sampling module 138may employ a machine learning technique (e.g., one-class SVM) toperiodically (e.g., every five minutes) generate the sample distancesignatures 130(1)-(n) using recently-collected data from the metricsinformation 128 over a preconfigured period of time, e.g., the precedingfour hours, and provide the same distance signatures 130(1)-(n) to thedetection module 140. As used herein, in some aspects, one-class SVM mayrefer to an unsupervised algorithm that learns a decision function fornovelty detection: classifying new data as similar or different to thetraining set. As described in detail herein, in some aspects, one-classSVM may be employed to encapsulate all instances of a same class in ahyperplane (e.g., plurality of support vectors), thereby providing aconcise representation (e.g., the reference distance signatures, 126 andthe sample distance signatures 130). Further, the baseline monitoringmodule 124 may compare the concise representations generated bydifferent one-class SVMs to determine an occurrence of a baselinechange. In some aspects, a distance between representations may be usedto identify a baseline change. E.g., if the distance is greater than thedistance threshold determined by the training module 136, the baselinemonitoring module 124 may determine that a baseline change has occurred.

In some aspects, the sampling module 138 may perform one or morepre-processing steps on the recently-collected metrics information 128prior to generating the sample distance signatures 130 for the componentmetrics. For example, the sampling module 138 may identify a time seriesfor a component metric within the metrics information 128, deseasonalizethe time series, and/or remove outliers within the time series that arebelow or above preconfigured thresholds. Further, the sampling module138 may scale the time series to a preconfigured range (e.g., between 0and 1). In addition, the sampling module 138 may employ a one-class SVMmodel with the pre-processed time series. For example, the samplingmodule 138 may determine a plurality of support vectors via a one-classSVM based on the pre-processed time series information, compute a matrixof pairwise distances between each pair of the support vectors, anddetermine a kernel of the matrix. In some aspects, the sampling module138 may employ gaussian RBF as the kernel function used to generate thekernel.

The detection module 140 may be configured to detect baselines changesof the component metrics by comparing a reference distance signature 126for a component metric to sample distance signature 130 for a componentmetric. In particular, the detection module 140 may identify anoccurrence of a baseline change based on a dissimilarity measure basedon the reference distance signature 126 and the sample distancesignature 130 being greater than a distance threshold (e.g., thedistance threshold determined for the component metric by the trainingmodule 136). For instance, in some aspects, the detection module 140 maycompute the dissimilarity measure between the regions of the referencedistance signature 126 and the sample distance signature 130 (e.g., thekernel matrix distance between the kernel generated by the trainingmodule 136 for the reference distance signature 126 and the kernelgenerated by the sampling module 138 for the sample distance signature130). In some aspects, determining the dissimilarity measure may requiren×m+c computations, where n and m are respectively the support vectorsof the train and the sample space and c a small constant. Given that mis usually very small (e.g., around 4-5 vectors), the detection module140 may efficiently perform inference in the detection phase.

In some aspects, the detection module 140 may be further configured toidentify an occurrence of a baseline change based on a median thresholdindicating that the baseline change has a predefined level of impact.For example, in response to the dissimilarity measure being greater thanthe distance threshold, the detection module 140 may further determine amedian context of the component metric, i.e., whether a median thresholdis applied as a ceiling or floor. For example, a component metricassociated with latency may have a median context requiring that thechange to the media of the baseline is above a preconfigured threshold(e.g., seventy percentile) while a component metric associated withavailability may have a median context requiring that the change to themedian of the baseline is below a preconfigured threshold (e.g., thirtypercentile). In some aspects, the preconfigured threshold and the mediancontext may be set using the configuration module 132. Once the mediancontext has been identified, the detection module 140 may be configuredto determine the median threshold, determine the median of the timeseries corresponding to the sample distance signature 130, and comparethe median to the median threshold. Further, in some aspects, thedetection module 140 may detect a baseline change based upon thedissimilarity measure being greater than the distance threshold and themedian corresponding to the sample distance signature 130 being greaterthan the median threshold applied as a floor or less than a mediathreshold applied as a ceiling.

Further, the incident management module 142 may be configured to providenotification of baseline change activity detected by the detectionmodule 140. For example, the incident management module 142 may receivean indication of a candidate baseline change for a tenant component116(1) over a particular period of time from the detection module 140,and transmit an alert notification 146 to a tenant device 108(1)associated with the tenant component 116(1). In some aspects, the alertnotification 146 may identify the tenant component 116(1).

FIG. 2A illustrates an example graphical representation 200 of abaseline change, in accordance with some aspects of the presentdisclosure. As illustrated in FIG. 2A, the graphical representation 200includes time series 202 that experiences a potential baseline change.As illustrated in FIG. 2 , a baseline change 204 may occur that does notinclude a spike above or below the anomaly thresholds 206-208. FIG. 2Billustrates an example graphical representation 210 of a referencedistance signature 212 and a sample distance signature 214, inaccordance with some aspects of the present disclosure.

Example Process

The processes described in FIG. 3 below are illustrated as a collectionof blocks in a logical flow graph, which represent a sequence ofoperations that can be implemented in hardware, software, or acombination thereof. In the context of software, the blocks representcomputer-executable instructions stored on one or more computer-readablemedia that, when executed by one or more processors, perform the recitedoperations. Computer-readable media includes computer storage media,which may be referred to as non-transitory computer-readable media.Non-transitory computer-readable media may exclude transitory signals.Storage media may be any available media that can be accessed by acomputer. By way of example, and not limitation, such computer-readablemedia can include a random-access memory (RAM), a read-only memory(ROM), an electrically erasable programmable ROM (EEPROM), optical diskstorage, magnetic disk storage, other magnetic storage devices,combinations of the aforementioned types of computer-readable media, orany other medium that can be used to store computer executable code inthe form of instructions or data structures that can be accessed by acomputer. Generally, computer-executable instructions include routines,programs, objects, components, data structures, and the like thatperform particular functions or implement particular abstract datatypes. The order in which the operations are described is not intendedto be construed as a limitation, and any number of the described blockscan be combined in any order and/or in parallel to implement theprocesses. The operations described herein may, but need not, beimplemented using the cloud computing platform 102. By way of exampleand not limitation, the method 300 is described in the context of FIGS.1-2 and 4 . For example, the operations may be performed by one or moreof the baseline monitoring module 124, the configuration module 132, themetrics measurement module 134, the training module 136, the samplingmodule 138, the detection module 140, and the incident management module142.

FIG. 3 is a flow diagram illustrating an example method 300 forevaluating peer groups for comparative anomaly, in accordance with someaspects of the present disclosure.

At block 302, the method 300 may include generating a reference distancesignature based on historic time series information for a componentmetric, the historic time series information corresponding to a firstperiod of time. For example, the training module 136 may generate areference distance signature 126(1) for a component metric of a tenantcomponent 116(1) of the cloud computing platform 102 based on historicinformation (e.g., the preceding week) within the metrics information128. In some other examples, the training module 136 may generate areference distance signature 126(1) for a component metric of a resource114 or a service 118 of the cloud computing platform 102. In someaspects, the component metric may measure one of a request duration, adependency duration, or client performance. Further, in some aspects,the reference distance signature 126(1) may be a one-class supportvector machine model.

Accordingly, the cloud computing platform 102, the cloud computingdevice 400, and/or the processor 402 executing the training module 136may provide means for generating a reference distance signature based onhistoric time series information for a component metric, the historictime series information corresponding to a first period of time.

At block 304, the method 300 may include generating a sample distancesignature based on sample time series information for the componentmetric, the sample time series information corresponding to a secondperiod of time. For example, the sampling module 138 may generate asample distance signature 130(1) for a component metric of a tenantcomponent 116(1) of the cloud computing platform 102 fromrecently-collected information (e.g., preceding four hours) from themetrics information. In some other examples, sampling module 138 maygenerate a sample distance signature 130(1) for a component metric of aresource 114 or a service 118 of the cloud computing platform 102.Further, in some aspects, the sample distance signature 130(1) may be aone-class support vector machine model.

Accordingly, the cloud computing platform 102, the cloud computingdevice 400, and/or the processor 402 executing the sampling module 138may provide means for generating a sample distance signature based onsample time series information for the component metric, the sample timeseries information corresponding to a second period of time.

At block 306, the method 300 may include comparing the referencedistance signature to the sample distance signature to determine asignature difference. For example, the detection module 140 may computethe dissimilarity measure between the regions of the reference distancesignature 126 and the sample distance signature 130 (e.g., the kernelmatrix distance between the kernel generated by the training module 136for the reference distance signature 126 and the kernel generated by thesampling module 138 for the sample distance signature 130).

Accordingly, the cloud computing platform 102, the cloud computingdevice 400, and/or the processor 402 executing the detection module 140may provide means for comparing the reference distance signature to thesample distance signature to determine a signature difference.

At block 308, the method 300 may include determining that the secondperiod of time is a baseline change candidate based on the signaturedifference being greater than a distance threshold. For example, thedetection module 140 may identify an occurrence of a baseline changebased on the dissimilarity measure based on the reference distancesignature 126 and the sample distance signature 130 being greater than adistance threshold (e.g., the distance threshold determined for thecomponent metric by the training module 136).

Accordingly, the cloud computing platform 102, the cloud computingdevice 400, and/or the processor 402 executing the detection module 140may provide means for determining that the second period of time is abaseline change candidate based on the signature difference beinggreater than a distance threshold.

At block 310, the method 300 may include presenting, based at least inpart on the signature difference, an alert notification identifying thesecond period of time as the baseline change candidate. For example, theincident management module 142 may be configured to provide notificationof baseline change activity detected by the detection module 140. Insome aspects, the incident management module 142 may be configured totransmit the alert notification 146 to a tenant device 108 or amanagement device 144 associated with the tenant component 116(1).

Accordingly, the cloud computing platform 102, the cloud computingdevice 400, and/or the processor 402 executing the incident managementmodule 142 may provide means for presenting, based at least in part onthe signature difference, an alert notification identifying the secondperiod of time as the baseline change candidate.

Additionally, or alternatively, in an aspect, the method 300 may furtherinclude generating initial deseasonalized time series information basedon the historic time series information, removing outlier datapointsfrom within the initial deseasonalized time series information todetermining updated deseasonalized time series information, the outlierdatapoints above a first outlier threshold or below a second outlierthreshold, scaling the updated deseasonalized time series information todetermine scaled historic time series information, removing one or moreoutages via an offline algorithm to determine formatted historic timeseries information, and generating, based on the formatted historic timeseries information, the reference distance signature via a one-classSVM. For example, the training module 136 may identify a time series fora component metric within the metrics information 128, deseasonalize thetime series to generate initial deseasonalized time series information,and/or remove outliers within the time series that are below or abovepreconfigured thresholds to generate the updated deseasonalized timeseries information. Further, the training module 136 may scale the timeseries to a preconfigured range (e.g., between 0 and 1) to generatescaled historic time series information, and/or remove one or moreanomalies and/or outages via an offline algorithm to generate theformatted historic time series information. Further, the training module136 may generate the reference distance signature 126 via a one-classSVM using the pre-processed time series (i.e., the formatted historictime series information).

Accordingly, the cloud computing platform 102, the cloud computingdevice 400, and/or the processor 402 executing the training module 136may provide means for generating initial deseasonalized time seriesinformation based on the historic time series information, removingoutlier datapoints from within the initial deseasonalized time seriesinformation to determining updated deseasonalized time seriesinformation, the outlier datapoints above a first outlier threshold orbelow a second outlier threshold, scaling the updated deseasonalizedtime series information to determine scaled historic time seriesinformation, removing one or more outages via an offline algorithm todetermine formatted historic time series information, and generating,based on the formatted historic time series information, the referencedistance signature via a one-class SVM.

Additionally, or alternatively, in an aspect, the method 300 may furtherinclude generating initial deseasonalized time series information basedon the historic time series information, removing outlier datapointsfrom within the initial deseasonalized time series information todetermining updated deseasonalized time series information, thedatapoints above a first outlier threshold or below a second outlierthreshold, scaling the updated deseasonalized time series information todetermine formatted historic time series information, and generating,based on the formatted historic time series information, the sampledistance signature via a one-class SVM. For example, the sampling module138 may identify a time series for a component metric within the metricsinformation 128, deseasonalize the time series to generate initialdeseasonalized time series information, and/or remove outliers withinthe time series that are below or above preconfigured thresholds togenerate the updated deseasonalized time series information. Further,the sampling module 138 may scale the time series to a preconfiguredrange (e.g., between 0 and 1) to generate the formatted historic timeseries information. Further, the sampling module 138 may generate thesample distance signature 130 via a one-class SVM using thepre-processed time series (i.e., the formatted historic time seriesinformation).

Accordingly, the cloud computing platform 102, the cloud computingdevice 400, and/or the processor 402 executing the training module 136may provide means for generating initial deseasonalized time seriesinformation based on the historic time series information, removingoutlier datapoints from within the initial deseasonalized time seriesinformation to determining updated deseasonalized time seriesinformation, the outlier datapoints above a first outlier threshold orbelow a second outlier threshold, scaling the updated deseasonalizedtime series information to determine formatted historic time seriesinformation, and generating, based on the formatted historic time seriesinformation, the sample distance signature via a one-class SVM.

Additionally, or alternatively, in order to generate the referencedistance signature the method 300 may further include determining afirst plurality of support vectors based on the historic time seriesinformation, and determining a first kernel of a matrix determined frompairwise distances between each pair of the first plurality of supportvectors. Accordingly, the cloud computing platform 102, the cloudcomputing device 400, and/or the processor 402 executing the trainingmodule 136 may provide means for determining a first plurality ofsupport vectors based on the historic time series information, anddetermining a first kernel of a matrix determined from pairwisedistances between each pair of the first plurality of support vectors.

Additionally, or alternatively, in order to generate the sample distancesignature the method 300 may further include determining a secondplurality of support vectors based on recently-collected time seriesinformation and determining a second kernel of a matrix determined frompairwise distances between each pair of the second plurality of supportvectors. For example, the sampling module 138 may determine a pluralityof support vectors via a one-class SVM based on the pre-processed timeseries information, compute a matrix of pairwise distances between eachpair of the support vectors, and determine a kernel of the matrix. Insome aspects, the sampling module 138 may employ gaussian RBF as thekernel function used to generate the kernel. Accordingly, the cloudcomputing platform 102, the cloud computing device 400, and/or theprocessor 402 executing the training module 136 may provide means fordetermining a second plurality of support vectors based onrecently-collected time series information and determining a secondkernel of a matrix determined from pairwise distances between each pairof the second plurality of support vectors.

Additionally, or alternatively, the method 300 may further includedetermining a plurality of time series windows within the historic timeseries information, determining a plurality of time series window scoresfor the plurality of time series windows, each window score representinga distance between an individual time series window and the plurality oftime series windows, and determining the distance threshold by applyinga three-sigma calculation to the plurality of time series window scores.For example, the training module 136 may be configured to partition atime series for a component metric of a cloud component into a pluralityof time series windows, and determine a distance score for each of theplurality of time series windows. In some aspects, each distance scorerepresents a distance between an individual time series window and theother plurality of time series windows. Further, the training module 136may determine the distance threshold by generating a vector of thedistance scores and applying a three-sigma calculation to the vector toidentify an abnormal score for the time series. Accordingly, the cloudcomputing platform 102, the cloud computing device 400, and/or theprocessor 402 executing the training module 136 may provide means fordetermining a plurality of time series windows within the historic timeseries information, determining a plurality of time series window scoresfor the plurality of time series windows, each window score representinga distance between an individual time series window and the plurality oftime series windows, and determining the distance threshold by applyinga three-sigma calculation to the plurality of time series window scores.

Additionally, or alternatively, in an aspect, the method 300 may furtherinclude determining a median context based on the component metric,identifying a median threshold associated with the median context, anddetermining a median value meets the median threshold, wherein topresenting the alert notification, comprises presenting the alertnotification based at least in part on the median value meeting themedian threshold. For example, the detection module 140 may beconfigured to determine the median context, determine the medianthreshold based on the media context, determine the median of the timeseries corresponding to the sample distance signature 130, and comparethe median to the median threshold. Further, in some aspects, thedetection module 140 may detect a baseline change based upon thedissimilarity measure being greater than the distance threshold and themedian corresponding to the sample distance signature 130 meeting themedian threshold (i.e., being greater than a median threshold with afloor as the median context, or being less than a median threshold witha ceiling as the median context).

Accordingly, the cloud computing platform 102, the cloud computingdevice 400, and/or the processor 402 executing the detection module 140may provide means for determining a median context based on thecomponent metric, identifying a median threshold associated with themedian context, and determining the median is meets the medianthreshold, wherein to present the alert notification, the at least oneprocessor is configured to present the alert notification based at leastin part on the median value meeting the median threshold.

While the operations are described as being implemented by one or morecomputing devices, in other examples various systems of computingdevices may be employed. For instance, a system of multiple devices maybe used to perform any of the operations noted above in conjunction witheach other.

Illustrative Computing Device

Referring now to FIG. 4 , an example of a cloud computing device(s) 400(e.g., cloud computing platform 102). In one example, the cloudcomputing device(s) 400 includes the processor 402 for carrying outprocessing functions associated with one or more of components andfunctions described herein. The processor 402 can include a single ormultiple set of processors or multi-core processors. Moreover, theprocessor 402 may be implemented as an integrated processing systemand/or a distributed processing system. In an example, the processor 402includes, but is not limited to, any processor specially programmed asdescribed herein, including a controller, microcontroller, a computerprocessing unit (CPU), a graphics processing unit (GPU), an applicationspecific integrated circuit (ASIC), a field programmable gate array(FPGA), a system on chip (SoC), or other programmable logic or statemachine. Further, the processor 402 may include other processingcomponents such as one or more arithmetic logic units (ALUs), registers,or control units.

In an example, the cloud computing device 400 also includes the memory404 for storing instructions executable by the processor 402 forcarrying out the functions described herein. The memory 404 may beconfigured for storing data and/or computer-executable instructionsdefining and/or associated with the operating system 406, the baselinemonitoring module 124, the configuration module 132, the metricsmeasurement module 134, the training module 136, the sampling module138, the detection module 140, the incident management module 142, andthe processor 402 may execute the operating system 406, the baselinemonitoring module 124, the configuration module 132, the metricsmeasurement module 134, the training module 136, the sampling module138, the detection module 140, and the incident management module 142.An example of memory 404 may include, but is not limited to, a type ofmemory usable by a computer, such as random access memory (RAM), readonly memory (ROM), tapes, magnetic discs, optical discs, volatilememory, non-volatile memory, and any combination thereof. In an example,the memory 404 may store local versions of applications being executedby processor 402.

The example cloud computing device 400 also includes a communicationscomponent 410 that provides for establishing and maintainingcommunications with one or more parties utilizing hardware, software,and services as described herein. The communications component 410 maycarry communications between components on the cloud computing device400, as well as between the cloud computing device 400 and externaldevices, such as devices located across a communications network and/ordevices serially or locally connected to the cloud computing device 400.For example, the communications component 410 may include one or morebuses, and may further include transmit chain components and receivechain components associated with a transmitter and receiver,respectively, operable for interfacing with external devices. In animplementation, for example, the communications component 410 mayinclude a connection to communicatively couple the client devices104(1)-(N) to the processor 402.

The example cloud computing device 400 also includes a data store 412,which may be any suitable combination of hardware and/or software, thatprovides for mass storage of information, databases, and programsemployed in connection with implementations described herein. Forexample, the data store 412 may be a data repository for the operatingsystem 406 and/or the applications 408.

The example cloud computing device 400 also includes a user interfacecomponent 414 operable to receive inputs from a user of the cloudcomputing device 400 and further operable to generate outputs forpresentation to the user. The user interface component 414 may includeone or more input devices, including but not limited to a keyboard, anumber pad, a mouse, a touch-sensitive display (e.g., display 416), adigitizer, a navigation key, a function key, a microphone, a voicerecognition component, any other mechanism capable of receiving an inputfrom a user, or any combination thereof. Further, the user interfacecomponent 414 may include one or more output devices, including but notlimited to a display (e.g., display 416), a speaker, a haptic feedbackmechanism, a printer, any other mechanism capable of presenting anoutput to a user, or any combination thereof.

In an implementation, the user interface component 414 may transmitand/or receive messages corresponding to the operation of the operatingsystem 406 and/or the applications 408. In addition, the processor 402executes the operating system 406 and/or the applications 408, and thememory 404 or the data store 412 may store them.

Further, one or more of the subcomponents of the baseline monitoringmodule 124, the configuration module 132, the metrics measurement module134, the training module 136, the sampling module 138, the detectionmodule 140, and the incident management module 142, may be implementedin one or more of the processor 402, the applications 408, the operatingsystem 406, and/or the user interface component 414 such that thesubcomponents of the baseline monitoring module 124, the configurationmodule 132, the metrics measurement module 134, the training module 136,the sampling module, the detection module 140, and the incidentmanagement module 142, are spread out between thecomponents/subcomponents of the cloud computing device 400.

Conclusion

In closing, although the various embodiments have been described inlanguage specific to structural features and/or methodological acts, itis to be understood that the subject matter defined in the appendedrepresentations is not necessary limited to the specific features oracts described. Rather, the specific features and acts are disclosed asexample forms of implementing the claimed subject matter.

1-20. (canceled)
 21. A system comprising: a processor; and memorycomprising computer executable instructions that, when executed, performoperations comprising: generating a reference distance signature basedon historic time series information for a component metric, the historictime series information corresponding to a first period of time, whereingenerating the reference distance signature comprises: generatinginitial deseasonalized time series information based on the historictime series information; determining updated deseasonalized time seriesinformation by removing outlier datapoints from the initialdeseasonalized time series information; determining formatted historictime series information by removing one or more outages; and generatingthe reference distance signature based on the formatted historic timeseries information; generating a sample distance signature based onsample time series information for the component metric, the sample timeseries information corresponding to a second period of time; determininga signature difference by comparing the reference distance signature tothe sample distance signature; determining that the second period oftime is a baseline change candidate based on the signature differencebeing greater than a distance threshold; and presenting an alertnotification identifying the second period of time as the baselinechange candidate.
 22. The system of claim 21, further comprising: acloud computing platform including a plurality of tenant components. 23.The system of claim 22, wherein the component metric measures activityfrom a tenant component in the plurality of tenant components.
 24. Thesystem of claim 23, wherein the component metric corresponds to one of:request duration metrics; dependency duration metrics; or clientperformance metrics.
 25. The system of claim 24, wherein the requestduration metrics represent an amount of time elapsed between receipt ofa service request at the tenant component and transmission of a serviceresponse by the tenant component.
 26. The system of claim 24, whereinthe client performance metrics represent an amount of time elapsedduring completion of a process by the tenant component for a type of aclient.
 27. The system of claim 26, wherein the type of the clientcorresponds to: a browser; an operating system; or a client application.28. The system of claim 21, further comprising: a training module usedto periodically generate reference distance signatures for the componentmetric.
 29. The system of claim 28, wherein the training module uses aone-class support vector machine (SVM) to generate the referencedistance signature based on historic data collected over a preconfiguredperiod of time.
 30. The system of claim 29, wherein using the one-classSVM to generate the reference distance signature comprises scaling theupdated deseasonalized time series information to a preconfigured range.31. The system of claim 29, wherein the training module: determines aplurality of support vectors via the one-class SVM based onpre-processed time series information; computes a matrix of pairwisedistances between each pair of support vectors in the plurality ofsupport vectors; and determines a kernel of the matrix.
 32. The systemof claim 31, wherein the training module determines the kernel of thematrix using a gaussian radial basis function (RBF) as a kernel functionused to generate the kernel.
 33. The system of claim 21, furthercomprising: a sampling module used to periodically generate sampledistance signatures for the component metric.
 34. The system of claim33, wherein the sampling module uses a one-class support vector machine(SVM) to generate the sample distance signature based onrecently-collected data collected over a preconfigured period of time.35. The system of claim 34, wherein the one-class SVM encapsulatesinstances of a same class in a hyperplane to provide a representation ofthe reference distance signature and the sample distance signature. 36.A method comprising: a processor; and memory comprising computerexecutable instructions that, when executed, perform operationscomprising: generating a reference distance signature based on historictime series information for a component metric, the historic time seriesinformation corresponding to a first period of time; generating a sampledistance signature based on sample time series information for thecomponent metric, the sample time series information corresponding to asecond period of time, wherein generating the sample distance signaturecomprises: generating initial deseasonalized time series informationbased on the sample time series information; determining updateddeseasonalized time series information by removing outlier datapointsfrom the initial deseasonalized time series information; determiningformatted sample time series information by scaling the updateddeseasonalized time series information; and generating the sampledistance signature based on the formatted sample time seriesinformation; determining a signature difference by comparing thereference distance signature to the sample distance signature;determining that the second period of time is a baseline changecandidate based on the signature difference being greater than adistance threshold; and presenting an alert notification identifying thesecond period of time as the baseline change candidate.
 37. The methodof claim 36, wherein the signature difference represents a kernel matrixdistance between a first kernel associated with the reference distancesignature and a second kernel associated with the sample distancesignature.
 38. The method of claim 36, wherein the distance thresholdindicates that a baseline change has a predefined level of impact. 39.The method of claim 36, wherein the alert notification is provide to atenant device associated with a tenant component of a cloud computingplatform, the tenant component being a website, the component metricmeasuring activity of the website.
 40. A device comprising: a processor;and memory comprising computer executable instructions that, whenexecuted, perform operations comprising: generating a reference distancesignature based on historic time series information for a componentmetric, the historic time series information corresponding to a firstperiod of time, the reference distance signature being generated via afirst support vector machine (SVM) model; generating a sample distancesignature based on sample time series information for the componentmetric, the sample time series information corresponding to a secondperiod of time, the sample distance signature being generated via asecond SVM model; determining a signature difference by comparing thereference distance signature to the sample distance signature;determining that the second period of time is not a baseline changecandidate based on the signature difference being less than a distancethreshold; and providing an indication that the second period of time isnot the baseline change candidate.